Model-Free Optimal Control of Linear Multiagent Systems via Decomposition and Hierarchical Approximation

Designing the optimal linear quadratic regulator (LQR) for a large-scale multiagent system is time consuming since it involves solving large-size matrix Riccati equation. The situation further exasperated when design needs to be done in model-free way using schemes such as reinforcement learning (RL). To reduce this computational complexity, we decompose LQR problem into multiple small-size problems. We consider objective function specified over an undirected graph, and cast decomposition graph clustering problem. decomposed two parts, one consisting of independent clusters connected components, other containing edges that connect different clusters. Accordingly, resulting controller has hierarchical structure, components. first component optimizes performance each cluster by RL algorithm. second accounts coupling clusters, which achieved least-squares shot. Although suboptimal, adheres particular structure interagent couplings strategy. Mathematical formulations are established find minimizes number required communication links or reduces optimality gap. Numerical simulations provided highlight pros cons proposed designs.